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ABSTRACT 

An  evaluation  of  several  of  the  various  methods  available  for 
analyzing  multiprogramming  scheduling  algorithms  is  conducted.    The 
evaluation  consists  of  an  analysis  of  the  capabilities  and  limitations 
of  the  methods  and  an  examination  of  their  effectiveness,  when  used  on 
three  different  types  of  scheduling  algorithms .    The  three  types  of 
algorithms  are:    (1)  a  single-level  round-robin  type,  (2)  a  multi-level 
dynamic  priority  allocation  scheme,  and  (3)  a  three-level  implicit 
priority  method  proposed  by  the  author  for  general  application,  including 
sample-data  control  use.    The  results  of  the  evaluation  are  used  in 
formulating  a  composite  investigation  procedure  that  contains  the  best 
features  of  the  various  methods  of  analysis  „ 

The  author  wishes  to  express  his  appreciation  to  Professor 
Mitchell  L.  Cotton  of  the  U.S.  Naval  Postgraduate  School  for  his 
assistance  and  encouragement  during  this  investigation. 
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1 . 0      Introduction 


The  increasing  demands  upon  computing  facilities  have  brought 
about  the  need  for  more  effective  usage  of  present  and  proposed  facilities  . 

As  the  number  of  users  and  user  applications  grow,  it  becomes 
apparent  that  a  departure  from  the  normal  type  of  design  and  usage  will 
be  necessary,  in  order  to  meet  expected  future  demands  upon  computing 
facilities.    This  is  true  in  the  case  of  some  facilities  even  now. 

A  solution  to  this  problem  is  the  multiprogramming  type  of  system 
design  along  with  closer  coupling  between  man  and  the  computer.    Multi- 
programming is  defined  by  Critchlow  (6)  as,  "The  time-sharing  of  a  processor 
by  many  programs  operating  sequentially."    This  means  that  many  object 
programs  are  in  memory  (core  or  auxiliary) ,  but  only  one  program  at  a  time 
is  actually  being  executed. 

The  term  multiprocessing  is  sometimes  confused  with  multiprogramming 
even  though  the  two  are  generally  regarded  as  distinct  concepts.    For  the 
purpose  of  this  paper  multiprocessing  will  mean,  the  time- sharing  of  the 
facilities  of  a  computer  by  many  programs  or  parts  of  the  same  program 
with  the  aim  of  achieving  maximum  use  of  each  and  every  facility.    Multi- 
programming only  will  be  considered  in  this  paper. 

The  control  of  the  object  programs  utilizing  a  multiprogramming 
system  is  the  job  of  a  supervisory  control  program,  which  is  generally 
called  the  executive  routine.    The  main  tasks  of  the  executive  routine  are: 
(1)  The  determining  of  which  object  program  to  execute  next  and  for  how 
long. 
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(2)  The  exchanging  of  object  programs  between  primary  and  secondary 
memory . 

(3)  The  control  of  all  I/O  operations . 

(4)  The  protection  of  all  portions  of  memory,  belonging  to  one  program , 
from  harm  by  another  program. 

The  efficient  operation  of  any  computer  system  depends  to  a  large 
extent  on  the  efficiency  and  correctness  of  its  executive.    This  is  even 
more  essential  when  applied  to  multiprogramming  systems,  where  each 
and  every  minute  operation  of  all  of  the  user  programs  must  be  scheduled 
and  made  to  occur  in  a  particular  sequence  that  will  maximize  the  systems 
effectiveness . 

A  portion  of  the  executive  must  be  allocated  the  task  of  scheduling 
these  operations  according  to  some  algorithm,  that  will  give  the  optimum 
system  performance  consistent  with  the  systems  goals .    When  used  in 
this  paper  scheduling  will  mean,  the  determination  of  the  sequence  in 
which  job  programs  will  use  the  available  facilities  of  the  computer 
system. 

This  paper  will  investigate  some  of  the  methods  of  analyzing  the 
operation  of  a  scheduling  algorithm,  point  out  their  capabilities  and 
shortcomings ,  and  propose  a  general  scheme  for  combining  the  better  of 
the  methods  into  an  organized  analysis  of  an  algorithm.    This  general 
scheme  of  analysis  will  enable  a  scientific  determination  of  the  proper 
scheduling  algorithm  for  a  multiprogramming  system. 


The  investigation  will  consist  of  a  general  examination  of  each 
method  of  analysis  and  examples  of  the  use  of  the  heuristic  and  statistical 
methods  on  several  representative  types  of  scheduling  algorithms.    The 
statistical  method  will  utilize  a  systems  simulator  based  on  the  Monte- 
Carlo  Method  o 


2  .0      Multiprogramming  Background 

The  terms,  time-sharing  and  multiprogramming,  are  generally  used 
in  such  a  manner  as  to  imply  a  common  definition .    This  definition  is 
taken  to  be  the  one  given  earlier  for  multiprogramming.    For  the  purposes 
of  this  paper  time-sharing  will  be  used  to  imply  the  sharing  of  a  facility 
of  the  computer  system  by  many  programs,  on  a  time  basis,  and  thereby 
will  become  a  tool  or  subset  of  multiprogramming  „ 
2.1      Motivations  for  use  of  Time-Sharing 

There    exist  many  motivations  for  the  use  of  time-sharing  in  a 
computer  system.    Some  of  which  are: 

(1)  Better  man-machine  interaction. 

(2)  More  efficient  usage  of  the  computer  system. 

(3)  Faster  turn  around  time  (the  time  between  formulation  of  a  program 
and  return  of  the  results)  „ 

Foremost  among  these  is  the  desire  and  need  for  a  much  faster 
man-machine  interaction  rate.      Computational  speed  has  been  steadily 
increasing,  while  the  methods  and  speeds  of  interaction  between  the 
user  and  the  computer  have  remained  relatively  unchanged.    The  desire 
to  increase  the  man-machine  interaction  rate  stems  from  the  philosophy 
that  every  man-made  system  is  designed  to  be  a  "mechanical  extension" 
of  man  and  help  him  in  the  performance  of  his  work.    Obviously,  the 
closer  the  coupling  between  the  system  and  man,  the  more  efficiently 
and  accurately  he  is  able  to  utilize  the  capabilities  of  the  system. 


The  normal  type  of  computer  system  processes  data  and  solves 
preformulated  problems  according  to  predetermined  procedures .    During 
the  course  of  computation  all  alternatives  must  be  considered  and  covered 
or  else  the  program  may  not  operate  properly  and  erroneous  results  may  be 
obtained. 

This  is  a  very  exhaustive  and  lengthy  program  formulation  process, 
which  may  consist  of  many  trial  runs  on  the  computer  before  all  alterna- 
tives are  covered  and  the  program  is  properly  formulated.    It  would  be  much 
easier  and  faster  if  the  user  could  utilize  the  capabilities  of  the  computer 
in  the  formulation  of  his  program.    The  formulation  phase  could  then  take 
the  form  of  a  guided  trial-and-error  process  in  which  the  computer  would 
turn  up  flaws  in  reasoning  or  reveal  unexpected  alternative  paths  in  the 
flow  of  the  program. 

By  time-sharing  the  computer  between  several  users  of  the  program 
formulation  type ,  not  only  would  the  rate  of  man-machine  interaction  be 
increased  but  the  utilization  of  the  computer  would  be  increased  also,  as 
any  "dead-time"  (unused  computer  time) ,  brought  about  by  one  user 
analyzing  a  computer  output  to  him  or  formulating  his  next  input  to  the 
computer,  could  be  utilized  by  another  user„ 

Any  one  of  the  above  mentioned  motivations  would  seem  to  be 
justification  for  a  multiprogramming  system  and  certainly  a  system  that 
could  accomplish  all  three  should  be  very  worthwhile  indeed . 
2.2      System  Goals 

Multiprogramming,  by  its  definition,  covers  a  vast  area  in  the 


computing  field.    Almost  any  computer  system  that  time  shares  any  of 
its  equipment  or  units  among  many  programs  may  lay  claim  to  being  a 
multiprogramming  system  of  a  sort      These  various  types  of  multiprogram- 
ming are  best  segmented  into  groups  according  to  the  goals  of  each  system 
There  exist  several  fairly  well  defined  goals  and  many  combinations  of 
these  with  which  to  segment  the  types  .    Some  examples  of  these  goals 
are: 

(1)  Providing  the  required  service  to  as  many  users  as  possible  is  one  of 
the  basic  goals  ,  which  defines  a  type  of  multiprogra mming  system  that  is 
receiving  more  and  more  attention  each  day.    This  is  as  it  should  be  for 
the  computer  system  is  a  man  made  tool  and  exists  for  the  purpose  of 
helping  man  solve  his  problems  .    Any  method  of  providing  better  service 
to  man  by  one  of  his  tools  should  receive  a  maximum  of  attention  and 
effort . 

(2)  Providing  the  maximum  utilization  of  each  and  every  unit  or  facility 
of  the  computer  system  in  such  a  manner  as  to  maximize  the  efficiency 
of  each  unit  and  that  of  the  system  as  a  whole .    The  efficiency  of  the 
computer  system  is  a  very  worthy  consideration,  especially  with  the 
vast  increase  in  the  usage  of  computers  that  is  occuring  today,    Com- 
puter systems  now  have  to  service  more  users ,  who  in  turn  utilize 
larger  and  more  complex  programs  than  were  required  a  few  years  back. 
This  necessitates  the  fullest  use  of  all  the  facilities  of  the  computer 
system. 


(3)  Providing  the  required  service  to  the  control  type  of  users  is  in 
actuality  a  subset  of  the  first  example,  but  is  of  such  importance  as  to 
merit  its  own  separate  mention.    This  type  of  multiprogramming  is  seeing 
more  and  more  applications  in  the  military  and  industrial  fields .    The 
problems  involved  are  difficult  and  will  require  much  effort  and  study. 
For  example,  how  does  a  computer  system  provide  guaranteed  response 
at  specific  times  on  a  regularly  recurring  basis  to  many  users?    This 
question  needs  to  be  answered  in  order  for  this  type  of  multiprogramming 
to  be  fully  realized. 

The  determination  of  goals  of  a  multiprogramming  system  is  of 
great  concern  in  the  formulation  of  the  executive  routine  and  in  fact 
specifies  the  overall  picture  of  the  results  that  the  executive  routine 
must  obtain. 
2.3      Scheduling 

While  multiprogramming  appears  to  go  a  long  way  toward  solving 
some  of  the  computer  usage  problems  ,  it  contains  many  problem  areas 
that  will  require  much  thought  and  effort  within  its  own  framework  . 
Some  of  these  are  outlined  by  Corbato  (4)s 

(1)  Scheduling  algorithms. 

(2)  Switching  versus  swapping  of  programs. 

(3)  Storage  allocation. 

(4)  Memory  and  input/output  protection . 

(5)  Secondary  memory  usage  schemes. 


Each  of  these  are  necessary,  integral  and  interdependent  sections 
of  a  multiprogramming  system.,    The  inadequacy  of  any  section  can  drasti- 
cally lower  the  effectiveness  of  the  system,  regardless  of  how  perfect  the 
other  sections  perform.    Of  prime  importance,  however,  is  the  scheduling 
section  of  the  multiprogramming  system's  executive  routine . 
2o3,l     System  goals  as  design  considerations 

Because  the  scheduler  decides  which  user  shall  receive  what  service 
and  when  it,  of  necessity,  must  clearly  reflect  the  goal  or  goals  of  the 
system.    In  fact,  it  may  be  considered  to  state  the  goals  in  its  algorithm. 
For  example,  if  the  scheduler  considers  only  the  fact  that  a  user  or  users 
are  requesting  service  and  honors  them  in  some  manner  that,  does  not 
depend  on  who  the  user  is  or  where  it  came  from,  it  then  must  have  as 
its  goal  the  optimization  of  service  to  all  users  without  regard  to  priorities . 
This  type  of  system  would  not  be  good  for  control  type  user  programs,  nor 
would  it  accomplish  much  toward  optimizing  the  computer  system  efficiency 
Consider  though,  the  system  that  is  concerned  with  computer  system  effi- 
ciency.   Its  scheduler  would  have  to  consider  the  past,,  present,  and  if 
possible,  future  work  load  of  each  major  unit  of  the  computer  system.    It 
would  have  to  determine  the  best  mix  of  jobs  or  job  segments  that  would 
keep  each  unit  as  busy  as  possible. 

It  is  apparent  then,  that  a  very  necessary  prelude  to  the  design  of 
a  scheduling  section  for  a  multiprogramming  system's  executive  routine, 
is  a  careful  and  precise  determination  of  the  goals  of  the  system..  When 
this  is  done  the  task  of  the  scheduler,  in  most  cases,  will  be  outlined 
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and  only  the  problem  of  accomplishing  it  remains      As  ir  the  case  of  much 

of  scientific  design,  the  answer  to  the  question,   "What  exactly  is  the 

problem,"  is  not  only  the  first,  but  sometimes  the  most  difficult  task 

encountered. 

2.3.2    Overhead  Considerations 

The  justification  of  the  overhead  created  by  any  section  of  the 
executive  routine  is  a  very  critical  factor.    It  is  even  of  a  more  critical 
nature  for  the  scheduler  than  for  other  sections,  as  the  scheduler  is  used 
more  often  than  the  other  sections  of  the  executive  routine  „    It  serves  no 
purpose  to  design  a  scheduler  that  considers  every  aspect  of  the  data 
concerning  the  user  programs  and  the  units  of  the  system,  calculates  the 
effect  on  the  realization  of  the  goals  of  the  system  and  goes  through  a 
complicated  process  to  maximize  the  realization,  only  to  the  end  of  taking 
up  more  computer  time  for  the  process  than  was  saved  (either  for  the  user 
or  the  system) .    This  is  a  clear  case  of  too  much  overhead  and  cannot  be 
tolerated.    The  ambitions  of  the  scheduler  should  be  lowered,  in  this  case, 
and  a  level  reached,  so  that  the  savings  more  than  exceed  the  overhead „ 
2.3<>3    Scheduler  Input  Considerations 

Another  problem  area  in  the  design  of  a  scheduler  is  the  inp<4f  infor- 
mation  required  for  the  operation  of  the  scheduler.    This  must  be  kept 
w  ithin  reasonable  bounds  as  some  information  requirements  may  be  either 
impossible  or  impractical.,    For  instance  it  would  be  desirable  to  have 
the  expected  running  time  of  each  user  program  requesting  service,  but 
this  fact  is  not  known  unless  the  program  has  been  run  before  with  the 


same  input  parameters  ,  a  rare  occurrence  indeed  „ 

2.3.4  Cost-effectiveness  Considerations 

In  all  aspects  of  the  design  of  a  scheduler  a  cost   effectiveness 
study  must  be  made  and  the  question  ,   "What  does  this  feature  buy  and 
cost  the  system  and/or  user?'1,  must  be  answered ,    The  cost-effectiveness 
criterion  as  applied  to  this  answer  will  determine  whether  or  not  the  feature 
should  be  adopted.    The  cost-effectiveness  criterion  is  another  Item  that 
is  very  dependent  upon  the  goals  of  the  system  .    The  system  that  is 
oriented  towards  the  control  type  user,  for  example ,  will  have  a  cost- 
effectiveness  criterion  that  demands  the  adoption  of  any  feature  enhancing 
the  system's  ability  to  guarantee  the  user  service  at  a  specific  sampling 
rate . 

2.3.5  Summary  of  General  Scheduler  Design  Considerations 

A  brief  summary  of  the  generalized  scheduler  design  problem  is? 

(1)  There  are  many  various  types  of  schedulers  and  each  is  specified  by 
the  particular  goals  of  the  overall  system  it  serves  „  Therefore,  specify 
the  goals  of  the  system  concisely  enough  and  the  type  of  scheduler  and 
its  task  is  specified  „ 

(2)  The  overhead  of  a  scheduler  must  not  be  allowed  to  get  out  of  hand 
and  the  complexity  of  the  scheduler  decreased  if  it  does. 

(3)  Input  requirements  to  the  scheduler  should  be  carefully  examined  for 
practicality  and  necessity. 

(4)  A  cost-effectiveness  study  should  be  made  of  every  feature  proposed 

for  the  scheduler  before  its  acceptance.    The  cost -effectiveness  criterion 

must  reflect  the  goals  of  the  system. 
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3.0  Investigation 

The  practical  aim  in  investigating  a  scheduling  system  by  any  forrr 
of  analytical  method  is  usually  to  analyize  the  actions  of  the  system  and 
try  to  improve  it  by  appropriate  changes .    For  example,  the  rate  of  arrival 
of  user  requests  may  be  so  high  that  large  queues  develop,  resulting  in 
long  delays  in  service  for  the  users,  or  the  rate  of  arrival  requests  may 
be  so  low  that  the  facilities  of  the  computer  system  are  idle  for  a  large 
portion  of  the  time.    A  change  in  the  scheduling  algorithm  may  be  indicated 
in  either  case.    Or  it  may  be  that  a  change  in  the  scheduling  algorithm  is 
being  contemplated  and  predictions  of  the  amount  of  change  in  service 
capabilities  are  needed.    In  any  event  frequent  analysis  of  the  operation 
of  a  scheduler  is  needed  to  insure  its  continuing  operation  at  optimum 
level . 

3.1  Methods  of  Investigation 

There  are  several  methods  of  investigating  the  operations  of  a 
scheduler  available  to  the  systems  designer,  each  of  which  has  its  own 
particular  areas  of  maximum  effectiveness.    These  methods  may  be  divided 
into  the  following  categories: 

(1)  Heuristic  approach. 

(2)  Queuing  theory  approach . 

(3)  Statistical  simulation  approach. 

(4)  Actual  system  implementation  approach. 
3.1.1    Heuristic  Approach 

The  heuristic  approach  generally  consists  of  a  mathematical  and 

logical  examination  of  the  operation  of  a  scheduler  without  the  necessity 
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of  gathering  large  amounts  of  data  for  analysis  .    As  a  result,  it  is  easier 
to  accomplish,  but  is  not  as  rigorous  as  other  methods,  nor  does  it  give  as 
complete  picture  of  the  operations.    It  may,  however,  turn  up  many  problem 
areas  and  indicate  possible  directions  for  further  investigation. 

3.1.2  Queuing  Theory  Approach 

The  queuing  theory  approach  is  an  analytical  method  that  furnishes 
theoretical  predictions  of  the  operation  of  the  scheduler.    These  predictions 
are  based  on  a  stochastic  model  of  the  scheduler  composed  of  mathematical 
formulas  derived  from  the  probabilistic  nature  of  the  scheduler  environment. 
This  model  requires  inputs  of  a  statistical  nature  and  therefore  necessitates 
the  investigation  of  fairly  large  amounts  of  data  concerning  the  system. 
For  example,  the  arrival  rate  pattern  of  user  requests,  the  service  rate 
pattern,  the  rules  governing  service  and  the  rules  governing  the  queue 
discipline.    The  basic  weaknesses  of  the  queuing  theory  approach  lie  in 
the  assumptions  that  sometimes  have  to  be  made  concerning  the  exact 
probabilistic  nature  of  the  environment  and  its  inability  to  handle 
scheduling  algorithms  that  specify  much  interdependence  between  the 
various  patterns  mentioned  above. 

3.1.3  Simulation  Approach 

The  simulation  approach  is  more  of  a  representation  of  the  behavior 
of  the  scheduler  than  a  mathematical  analysis,  as  is  the  case  with  the 
heuristic  and  queuing  theory  approaches.    The  simulation  approach  does, 
however,  utilize  the  theory  of  probability  and  statistics  in  simulating 
the  environment  of  the  scheduler. 
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The  simulator  is  an  ideal  method  to  use  in  place  of  queuing  theory 
when  there  exists  much  interdependence  between  the  parameters  of  the 
scheduler  environment.    The  type  of  detailed  results  obtained  from  simu- 
lation will  give  a  direct  qualitative  impression  of  what  the  behavior  of 
the  system  should  look  like . 
3.1.4    System  Implementation  Approach 

The  actual  implementation  approach  consists  of  placing  the  scheduling 
section  in  its  entirety  within  the  actual  multiprogramming  executive  routine 
and  testing  its  operation  within  the  complete  system.    This  method  is  not 
to  be  recommended  as  a  first  method  of  investigation  of  a  scheduler  with 
new  changes  as  it  is  extremely  costly  in  time  and  effort  while  also  causing 
the  disruption  of  the  normal  operation  of  the  system  during  the  testing  phase, 
3.2      Proposed  Composite  Investigation  Procedure 

Each  of  the  previously  mentioned  methods  has  its  own  place  in  the 
overall  procedure  of  the  investigation  of  a  scheduler  and  contributes  its 
particular  type  of  information  to  the  total  collection  of  intelligence  gleaned 
from  the  investigation . 

An  outline  of  such  an  overall  investigation  procedure  is  as  follows: 

(1)  Preliminary  survey. 

(2)  Collection  of  statistical  data. 

(3)  Detailed  study  of  the  system  and  effects  of  any  changes  to  the  system. 

(4)  Small  scale  trial  of  n^wly  proposed  system. 

(5)  Full  scale  implementation  of  proposed  system. 
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3.2.1  Preliminary  Survey 

A  preliminary  survey  of  a  scheduling  algorithm  would  entail  a  brief, 
but  careful  study  of  the  algorithm  and  its  basic  philosophy.    This  study 
would  be  of  the  heuristic  type  and  should  reveal  the  limitations  of  the 
algorithm  and  point  to  avenues  of  further  and  more  detailed  investigation, 
if  not  to  possible  modifications  themselves.    If  modifications  are  indicated, 
they  should  be  roughly  assessed  and  if  of  merit  passed  on  for  a  more  com- 
prehensive investigation. 

3.2.2  Collection  of  Statistical  Data 

The  next  step  involves  the  collection  of  statistical  data  for  use  with 
a  queuing  theory  analysis  of  the  scheduler  or  a  simulation  model  of  the 
scheduler. 

The  statistical  data  gathered  is  generally  segmented  as  follows: 

(1)  The  arrival  pattern  data  gives  the  statistical  pattern  of  the  arrival  of 
user  service  requests  and  is  generally  in  the  form  of  a  distribution  function, 
mean  arrival  rate  and  deviation  from  this  rate . 

(2)  The  service  mechanism  data  concerns  itself  with  the  number  of  users 
that  can  be  serviced  during  some  period  of  time  and  the  statistical  pattern 
of  the  length  of  service  time,  which  consists  of  a  distribution  function  and 
its  necessary  parameters  . 

The  arrival  pattern  data  may  be  separated  into  requests  for  various 
types  of  service  and  the  service  data  separated  in  an  analogous  manner . 
This  data  is  best  obtained  by  proper  use  of  statistical  sampling  methods . 
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3.2.3  Detailed  System  Study 

To  conduct  a  detailed  study  of  the  scheduler  and  its  operations 
normally  requires  the  use  of  a  simulation  model.  The  queuing  theory 
approach  will  in  general  be  much  harder  to  implement  and  not  give  as 
correct  data  due  to  the  complexity  and  interdependence  of  parameters 
found  in  most  scheduling  algorithms. 

The  correct  simulation  model  will  give  data  that  follows  closely 
the  operations  of  the  actual  system.    Thereby  allowing  an  investigation 
of  the  present  algorithm  to  be  made  under  easily  varied  conditions  and 
suggesting  modifications  that  may  be  checked  out  in  the  same  manner. 

3.2.4  Small  Scale  Trial 

The  modifications  that  have  passed  the  investigation  of  the  simula- 
tion model  should  then  be  incorporated  in  the  executive  routine  on  a  small 
scale  and  given  trial  runs  to  more  accurately  judge  their  compatability 
with  the  rest  of  the  system  and  their  effectivenesses.    This  procedure 
may  be  skipped  if  the  modifications  are  slight  and  it  is  readily  apparent 
that  they  will  be  easily  implemented  on  a  full  scale  basis  „    However, 
this  would  be  an  exception  and  time  and  effort  will  usually  be  saved  by 
a  small-scale  trial  run. 
3o2o5    Full  Scale  Implementation 

When  the  modifications  have  been  checked  out  by  a  small-scale  trial 
run  or  two,  they  should  be  incorporated  fully  into  the  executive  routine  and 
utilized  as  much  as  possible  for  many  hours  with  close  observation  and 
collection  of  data  in  order  to  judge  their  effectivenesses . 
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The  close  adherence  to  these  procedures  in  the  order  outlined,  will 
allow  the  adoption  of  a  new  scheduler  or  modification  of  an  old  one,  with 
full  assurance  that  it  will  not  disrupt  the  system  and  will  bring  about  a 
closer  realization  of  the  goals  of  the  system. 
3.3      Preliminary  Survey  by  Heuristic  Analysis 

As  stated  earlier  a  preliminary  survey  of  a  scheduling  algorithm  is 
a  necessary  prelude  to  further  investigation  of  a  new  algorithm  or  changes 
to  an  old  one.    This  survey  may  only  point  the  way  for  further  more  precise 
investigation  or  may  show  the  need  for  changes  by  itself.    In  most  cases, 
however ,  it  will  be  advisable  to  continue  the  investigation  with  more 
precise  methods . 

For  the  purpose  of  this  paper  a  preliminary  survey  of  several  different 
types  of  scheduling  algorithms  will  be  presented .    These  different  types  of 
algorithms  will  be  representative  of  algorithms  for  use  in  multiprogramming 
systems  each  having  goals  of  the  types  described  in  section  2„2„ 

No  attempt  will  be  made  to  prove  one  algorithm  better  than  another 
as  each  will  have  its  own  distinct  goals  and  will  be  designed  to  optimize 
them.    It  will,  instead,  be  the  aim  of  this  section  to  point  out  areas  of 
investigation  and  changes  that  would  appear  to  lead  to  a  better  realization 
of  the  systems  goals  or  a  broadening  of  the  goals  to  include  some  worth- 
while feature . 

The  algorithms  presented  will  generally  concern  themselves  more 
with  the  user's  interests  and  needs  rather  than  system  efficiency  as  this 
appears  to  be  of  the  major  concern  at  the  present  time„ 
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Three  algorithms  will  be  presented,  one  of  the  type  developed  by 
S.D.C  ,  one  proposed  by  Corbato  (4) ,  and  one  conceived  by  the  author. 
3.3.1    Algorithm  A 

One  of  the  first  categories  of  scheduling  algorithms  to  examine 
should  be  one  which  is  intended  for  use  in  a  multiprogramming  system 
with  optimization  of  user  service  as  its  goal.    An  algorithm  of  the  type 
used  by  System  Development  Corporation  within  their  "Time-Sharing 
System"  at  Santa  Monica,  California  is  an  excellent  example  of  a  member 
of  this  category.    This  algorithm  will  be  referred  to  as  Algorithm  A  here- 
after . 
3.3.1.1    Operation  of  Algorithm  A 

The  execution  of  many  user  programs  for  a  quantum  of  time  (q)   in  a 
round-robin  type  of  operation,  is  the  basic  principle  of  this  scheduling 
algorithm.    A  round-robin  queue  is  set  up  for  each  response  cycle  with 
the  users  in  the  queue  being  those  that  requested  service  during  the  last 
response  cycle  time.    The  quantum  or  time  of  operation  of  each  program 
during  the  cycle  is  determined  by  the  ratio  of  the  response  cycle  time  to 
the  number  of  users  in  the  queue.    The  cycle  time  (tr)  is  chosen  to  be 
approximately  the  average  human  response  time  in  order  to  achieve  the 
effect  of  simultaneous  use  of  the  computer  by  many  users . 

The  maximum  number  of  users  (Nmax)  is  determined,  in  part,  by  the 
minimum  allowable  quantum  time  (qmin) .    This  time  is  bounded  by  the 
amount  of  time  required  for  the  average  user  program  to  produce  a  response 
(I/O  operation) . 
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If  a  program  does  not  use  up  its  alloted  quantum  time  in  computation, 
but  is  terminated  by  a  I/O  or  finished  interrupt,  the  quantum  is  recalculated 
for  the  remaining  users  in  the  response  cycle.    The  redistributing  of  the 
remaining  time  in  the  response  cycle  not  only  increases  the  efficiency  of 
the  system,  but  also  allows  the  larger,  slower  user  programs  more  execution 
time.    If  at  any  time  during  the  cycle  no  users  are  in  the  queue,  the  system 
goes  into  an  "idle"  loop  and  waits  until  a  user  requests  service  (allowable 
during  every  interrupt  of  the  quantum  clock)  „ 

Priority  of  program  service  during  a  given  interval  of  time  is  deter- 
mined by  a  program Ds  size  and  its  use  of  low  speed  I/O  equipment.    The 
user  programs  occupy  different  banks  of  core  memory  according  to  their 
priorities.    The  priority  levels,  in  the  order  of  service,  are:  (l)  programs 
of  less  than  16K  and  not  using  low  speed  I/O,  (2)  programs  of  less  than 
16K  and  using  low  speed  I/O ,  or  programs  between  16K  and  32K  and  not 
using  low  speed  I/O ,   (3)  programs  in  excess  of  32K. 

This  scheduling  algorithm  is  well  suited  and  very  effective  for 
system  use  of  the  type  that  consists,  primarily, of  tasks  that  allow  a  high 
percentage  of  the  user's  system  time  to  be  taken  up  by  the  slow  human 
thought  and  reaction  process.    Examples  of  such  tasks  are:  (I)  on-line 
debugging,  (2)  on-line  calculation,  (3)  on-line  program  building  and  (4) 
war  games  . 

The  main  concern  of  a  scheduling  algorithm ,  when  dealing  with 
these  types  of  tasks,  is  to  furnish  the  users  with  reasonable  response  to 
service  requests  and  reasonable  reaction  times.    The  efficiency  of  the 
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computer  system  is  considered ,  but  not  allowed  to  become  of  primary 

importance . 

3.3.1.2    Effects  of  Varying  Basic  Parameters  of  Algorithm  A 

There  are  several  significant  parameters  that  may  be  examined 
during  a  preliminary  survey  of  this  algorithm  *    They  are:  (l)  maximum 
number  of  users,   (2)  response  times,   (3)  quantum  times  and  (4)  efficiency. 

By  the  use  of  variations  of  one  basic  formula  much  information  can 
be  obtained  concerning  these  parameters.    Th~  formula  is  of  the  "Worst- 
case"  type  in  that  it  gives  a  pessimistic  view  of  the  operations  based 
on  values  of  the  formula  parameters  that  represent  the  worst  conditions 
possible.    This  type  of  analysis  is  very  helpful,  as  it  shows  the  lower 
bound  of  service  capabilities  and  insures  good  operation  if  expectations 
do  not  exceed  this  lower  bound  by  significant  amounts . 

For  determining  the  maximum  number  of  users  allowable  with  the 
minimum  quantum,  with  the  percentage  of  overhead  (n)  and  the  "Worst- 
case"  value  of  the  time  it  takes  to  transfer  a  program  from  secondary  to 
primary  memory  or  vice  versa  (t£)  either  known  or  approximated ,  the 
following  formulas  apply. 
Equation  1  Equation  2 

Nmax  =    Srd-n)  Nmax  =     *r  (l-n) 

2t    +  qmin  2ts 

These  formulas  are  derived  by  analyzing  the  basic  cycle  of  operations 

concerning  one  user  during  a  response  cycle. 
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First  it  is  determined,  which  user  is  to  be  executed  for  the  particular 
quantum  of  time  and  his  program  is  brought  in  from,  secondary  memory, 
after  the  last  operating  program  is  transferred  to  secondary  memory  „    This 
takes  an  amount  of  time  equal  to  2t    „    The  program  is  then  operated  for 
one  quantum  (q) .    Of  course,  deciding  which  program  to  bring  in  and  other 
executive  functions  take  up  a  certain  percentage  of  the  cycle  time  (tj.)  and 
have  to  be  accounted  for  under  the  heading  of  overhead  (n)  „ 

Clearly,  then,  the  denominator  of  the  Equation  1  represents  the  time 
necessary  to  execute  one  user  during  the  response  cycle  while  the  numerator 
represents  the  effective  operation  time  available  during  the  response  cycle. 
It  must  be  emphasized  that  N max  is  the  maximum  number  of  active  users  in 
the  system  i.e.  ,  the  number  in  the  queue.    There  may  be  many  more 
actually  using  the  system  as  long  as  not  more  than  Nmax  request  service 
during  the  same  response  cycle . 

In  order  to  investigate  ways  of  increasing  Nmax,  the  effects  of 
changes  in  the  various  parameters  of  Equation  1  on  Nmax  must  be  analyzed. 
It  will  make  the  investigative  task  simpler,  if  the  parameters  are  broken 
up  into  groups  according  to  whether  they  are  easily  changed  by  a  software 
approach  or  require  hardware  modifications  to  induce  significant  changes. 

The  swap  time  (t  )  is  the  only  parameter  that  would  most  likely 
require  hardware  modifications  to  measurably  change  it.    For  instance, 
the  swap  time  may  be  decreased  and  Nmax  increased  by  increasing  the 
transfer  rate  between  primary  and  secondary  memory,  allowing  two  or  nore 
programs  in  memory,  or  both.    Increasing  the  transfer  rate  is  obviously  a 
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hardware  change  and  is  usually  a  major  one  if  substantial  increase  in 
rate  is  to  be  accomplished.    Allowing  two  programs  in  memory ,  at  first 
glance,  does  not  seem  to  require  hardware  modifications,  but  on  further 
analysis  it  will  be  seen  to  require  memory  protection  to  prevent  inter- 
action between  the  two  programs  and  dynamic  memory  relocatability ,  in 
order  to  achieve  the  desired  results. 

The  main  thought  concerning  hardware  changes  ,  whether  they  are 
modifications  to  existing  hardware  or  design  features  for  proposed  equip- 
ment, is  to  be  sure  that  the  increase  in  capability  is  substantial  enough 
to  warrant  the  cost  of  the  change. 

The  technique  of  showing  graphically  the  capability  increase  versus 
change  in  parameter  values  is  an  invaluable  tool  to  use  in  determining 
whether  or  not  further  study  is  warranted .    This  technique  is  illustrated 
in  Figure  1  where  curve  A  represents  Nmax  versus  primary  to  secondary 
memory  transmission  rate  and  curves  B  and  C  show  the  relationships 
between  qmin  and  tr,  respectively,  and  Nmax,    The  latter  two  variables, 
qmin  and  t  ,  are  variable  by  software  means  and  depend  more  on  user 
wants  and  needs  for  their  values  than  on  the  computer  system  limitations. 
This  means  that  the  cost  of  their  change  will  be  in  terms  of  changes  in 
some  user  service  parameters  rather  than  monetary  values  or  man-hours  „ 

The  estimation  of  the  effect  of  allowing  two  or  more  programs  in 
memory  at.  one  time  actually  involves  the  modification  of  Equation  1 .    If 
memory  sharing  is  allowed  and  qmin  is  substantially  less  than  2ts ,  then 
the  limiting  factor  in  the  denominator  of  Equation  I  becomes  2t    and 
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Equation  1  shifts  to  the  form  of  Equation  2  .    The  minimum  quantum  (qmin) 
effectively  vanishes  because  concurrency  is  now  possible  between  the 
transferring  of  one  program  and  the  execution  of  another.    If  qmin  were 
much  larger  than  2t    ,  then  2ts  would  likewise  effectively  vanish,  but  this 
will  hardly  ever  be  the  case. 

Whether  or  not  2t     is  substantially  larger  than  qmin  depends  upon 
the  distribution  of  user  requests  and  can  only  be  roughly  estimated  without 
such  data.    From  observations  of  the  "Time-Sharing  System"  at  S.D.C.  , 
2ts  does  not  appear  to  be  much  larger  than  qmin  and  therefore  the  effect 
of  memory  sharing  on  Nmax  does  not  seem  to  warrant  its  cost.    Of  course, 
this  applies  to  this  particular  system  and  its  users  and  the  outcome  of 
the  analysis  might  be  entirely  different  for  another  system  and  its  users. 
The  important  thing  is  that  the  method  of  analysis  will  remain  the  same . 

For  a  complete  analysis  of  the  algorithm,  other  pertinent  charac- 
teristics, such  as,  response  time  and  computational  efficiency,  must  be 
investigated  „ 

Response  time ,  for  this  algorithm,  may  be  considered  tr,  for  worst 
case  analysis.    In  other  words,  a  user  requesting  service  immediately 
after  a  new  response  cycle  had  started  would  have  to  wait  a  maximum  of 
time  (tr)  until  he  was  accepted  for  service,  while  one  which  requested 
service  at  the  end  of  a  cycle  would  have  to  wait  a  minimum:  of  time. 
Another  form  of  Equation  1  may  be  used  (Equation  3)  for  this  analysis 
and  graphical  results  illustrated  by  curve  C  of  Figure  1 . 
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Equation  3 

tr  =  N(2ts+q)/(l-n) 

As  may  be  seen  by  an  examination  of  the  equations  and  the  curves 

of  Figure  1 ,  N  and  t    are  linearly  dependent  (if  2t     and  q  remain  constant) 

r  s 

and  therefore  each  is  very  sensitive  to  any  change  in  the  other.    As  each 
increases  without  bound,  with  any  increase  in  the  other,  some  upper  limit 
must  be  set  for  one  of  them  in  order  to  contain  the  investigation  in  a 
workable  area.    This  limit  is  more  easily  placed  on  t  .    The  response  time 
should  not  be  allowed  to  exceed  a  value  that  would  appreciably  degredate 
the  service  needed  by  the  type  of  user  for  whom  the  system  has  been 
designed.    In  the  case  of  this  particular  algorithm,  the  maximum  t    is  an 
approximation  of  th<=  human  response  interval.    This  then  confines  the 
problem  to  a  smaller  working  area  and  families  of  curves  A,  B,  and  C  may 
be  drawn  within  this  area  for  closer  analysis. 

It  is  clearly  shown  by  the  graph  that  if  the  major  concern  is  to 
maximize  the  number  of  users,  then  t    must  be  set  at  its  maximum  value. 
On  the  other  hand,  if  servicing  a  user  with  particular  needs  as  to  response 
time  is  of  primary  interest  then  N  must  be  restricted  to  a  value  that  will 
allow  the  needed  t  .    In  most  cases  a  combination  of  these  goals  is  present 
and  a  compromise  is  reached  between  Nmax  and  t  max  with  the  other  para- 
meters being  varied  in  order  to  ease  the  conflicts  .    This  is  the  case  with 
the  particular  algorithm  in  question  as  t    is  restricted  to  a  maximum  value, 
but  allowed  to  vary  up  to  this  maximum  as  the  number  of  users  dictate. 

The  quantum  time  is  allowed  to  vary,  but  has  a  minimum  value  that 

is  normally  arrived  at  by  a  compromise  between  computational  efficiency 
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and  maximizing  the  number  of  users.    Clearly  as  q  is  decreased  the 
number  of  users  allowed  (N)  goes  up,  but  the  more  often  a  program  is 
transferred  in  and  out  of  memory  the  more  overhead  is  increased,  swap 

time  per  program  is  increased  and  computational  efficiency  is  decreased. 

. 

The  term  computational  efficiency  is  taken  to  be  a  measure  of  the  ratio  of 

er  pro 
the  time  a  user  program  spends  in  the  computational  phase  to  the  time  it 

spends  in  preparing  for  the  computational  phase.    The  latter  time  does  not. 

,  a  USe;  spends  i 

include  time  spent  waiting  for  a  normal  user  turn      The  following  equation 

is  one  method  of  expressing  computational  efficiency  (CtE„). 

lude  1 

Equation  4 

is  one 

C.E.  =  q(l-ng) 

2ts 
The  parameter  n    represents  the  portion  of  overhead  that  may  be 

H. 

considered  taken  up  by  each  user  everytim^  his  program  runs  for  a  quantum 

3  para: 
of  time.    Because  of  the  fact  that  ts  is  not  easily  variable  computational 

efficiency  depends  to  a  great  extent  on  quantum  time  and  is  maximum  when 

pi"  Because  of  the  fact  tha 

q  is  maximum. 

elf 

It  is  important  to  bear  in  mind  that  rarely  will  a  multiprogramming 

system  achieve  a  computational  efficiency  greater  than  50%  and  most  do 

not  even  reach  this.    Computational  efficiency,  therefore,  should  be 

maximized,  but  not  at  the  expense  of  user  service  and  should  not  be  used 

as  a  major  comparison  parameter  between  systems . 

The  minimum  quantum  time  is  determined  not  only  by  consideration 

of  computational  efficiency  (as  illustrated  in  Equation  4)  and  the  number 

of  users  allowed,  but  also  by  a  consideration  of  obtaining  the  maximum 
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use  out  of  every  assigned  quantum  of  time.    It  does  not  help  to  assign 
a  user  a  larger  quantum  of  time  in  order  to  gain  better  efficiency  only 
to  have  him  terminate  his  quantum  by  an  I/O  operation  after  only  a  few 
milliseconds.    Nor  does  it  help  to  assign  a  user  a  smaller  quantum  and 
have  him  need  only  a  little  more  to  come  to  a  logical  termination.    These 
situations  can  not  be  avoided  for  every  user,  but  by  a  careful  study  of 
user  distributions  as  far  as  program  run  time  between  I/O  operations  is 
concerned  and  anslysis  of  the  effects  of  varying  qf  a  suitable  compromise 
between  efficiency,  and  user  service  may  be  reached. 
3.3.1.3    Summary  of  Analysis  of  Algorithm  A 

The  following  comprise  a  brief  summary  of  the  heuristic  analysis 
of  Algorithm  A. 

(1)  Either  an  excellent  intuitive  feel  for  the  expected  distribution  of 

such  thing  as,  user  arrival  rates,  time  between  I/O  operations  and  response 
time  requirements  or  else  actual  distribution  data  should  be  available  for 
a  preliminary  analysis. 

(2)  The  user  service  oriented  scheduler  must  set  a  maximum  limit  on  tr 
that  is  based  primarily  on  user  response  requirements. 

(3)  A  minimum  limit  on  quantum  time  must  be  set  consistent  with  proper 
utilization  of  the  quantum  (based  on  user  time  between  l/0's  distribution)  „ 
number  of  users  allowed  in  each  response  cycle  and  computational  effi- 
ciency.   The  emphasis  should  be  placed  on  the  first  two  conditions. 

(4)  Unless  much  faster  transmission  rates  between  primary  and  secondary 
memory  are  available,  memory  sharing  does  not  appear  to  increase  the 
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capabilities  of  the  system  enough  to  warrant  its  implementation. 

(5)  A  simulation  study  of  the  effects  of  not  allowing  early  I/O  terminations 
to  cause  a  recalculation  of  q  would  be  helpful  in  determining  if  the 
overhead  involved  were  prohibitive . 

(6)  A  simulation  study  of  the  effects  on  computational  efficiency  and  user 
service  due  to  the  modification  of  the  algorithm  to  allow  background  jobs 
to  fill  in  when  the  system  has  no  other  users  would  also  be  advisable. 
3.3.2    Algorithm  B 

The  next  type  of  algorithm  presented  for  analysis  is  one  which  also 
has  as  its  goal  optimum  user  service,  but  does  give  consideration  to 
computational  efficiency. 

This  algorithm  (3)  was  developed  at  Massachusetts  Institute  of 
Technology  and  was  one  of  the  forerunners  of  the  field »    For  the  purpose 
of  this  paper  the  algorithm  will  be  designated  Algorithm  B  . 

Algorithm  B  is  what  is  considered  a  multilevel  scheduling  algorithm, 
as  the  scheduling  process  sets  up  multiple  level  queues  of  programs 
waiting  to  be  serviced. 
3.3.2.1    Operation  of  Algorithm  B 

The  multilevel  scheduling  algorithm  is  implemented  in  the  following 
manner: 

(1)  Each  user  program  is  assigned  to  the  L0  th  level  priority  queue,  as  it 
enters  the  system. 

Equation  5 
LQ=    ZLog2(/wp/wq_7   +  1)  J 
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w    =    number  of  words  in  user  program 

w    =    number  of  words  that  can  be  transferred  in  and  out  of  primary 
q 

memory  during  one  quantum  (q)  of  time 
l_K_£  Integral  part  of  A 
(2)  The  program  at  the  head  of  the  lowest  level  queue  (L)  is  executed  for  up 

to  2     quanta  of  time  and  if  it  is  not  completed  is  placed  at  the  end  of  the 

T  + 1 
L+ 1th  level  queue  to  be  executed  for  up  to  2         quanta  of  time  later  on 

L 
After  execution  of  all  of  the  Lt^  level  programs  for  2     quanta ,  programs  in 

the  L+lth  level  are  executed.    If  a  program  at  the  L+l+h  level  is  being 

executed  and  the  L.,    level  becomes  occupied  by  an  active  program ,  the 

presently  executing  program  is  placed  at  the  head  of  the  L+I^h  level  queue 

and  the  Lt^  level  program  is  brought  in  and  executed.,    If  there  are  no 

levels  active,  background  jobs  are  brought  in  and  executed  until  a  level 

becomes  active,  thereby  increasing  computational  efficiency  and  lowering 

idle  time . 

3.3.2.2    Effects  of  Varying  Basic  Parameters  of  Algorithm  B 

From  Equation  5  it  can  be  seen  that  program  size  is  an  essential 
input  into  Algorithm  B  and  therefore  requires  that  user  programs  be  pre- 
compiled . 

Further  study  of  the  equation  shows  that  L    can  never  be  less  than 
zero  due  to  the  fact  that  (wp/wq+I)  has  one  as  its  lower  limit.    With 
(w  /wq)  q  representing  the  swap  time  (2tg)  of  the  program,  then  L0  is 
always  at  least  slightly  greater  than  Log2  (2ts/q) .    This  coupled  with 
the  practice  of  executing  the  program  for  2Ij°(q)  or,  as 
(2Log2(2ts/q))  (q)  =  2ts,  for  greater  than  the  swap  time  of  the  program, 


insures  a  computational  efficiency  of  at  least  50%, 

A  computational  efficiency  of  at  least  50%  arises  from  this  analysis 
only  if  overhead  time  is  neglected.,    As  overhead  can  never  be  neglected, 
an  overhead  factor  (n)  must  be  included  in  any  equation  concerning 
efficiency  . 

Equation  6  may  be  taken  to  represent  computational  efficiency  with 
overhead  included  for  programs  operating  from  the  Lt^  level  queue., 

Equation  6 
C.E.  =Nq2L(l-n)   =  {  /wp/wq  J   +  1)  (l~n) 

N  (Zwp/wq-/)q  (ZVV    } 

Where  N  =  number  of  users  at  Level  L 

n  =  fraction  of  overhead  involved  for  each  user/quantum 
As  may  be  seen  Equation  6  reduces  to  the  point  where  computational 
efficiency  appears  to  be  independent  of  q  and  N  „    This  is  not  strictly  true 
as  n  is  a  proportional  parameter  and  is  based  on  a  particular  q„    This  then 
imposes  a  limit  on  q  that  requires  it  to  remain  large  enough  to  keep  the 
overhead  to  a  low  proportion  of  q  and  computational  efficiency  close  to 
50%, 

Another  equation  that  is  helpful  in  analyzing  Algorithm  B  is  one 
that  describes  the  calculation  of  the  worst-case  response  time  (tr)  for 
each  user  when  all  N  users  are  on  one  level. 

Equation  7 
tr  =  2Nq  (  Z  wp/wq_7  +  U    -  Nqn 
=  Nq        2  CZwp/wq_7+    L)  -  n 
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Equation  7  is  arrived  at  by  considering  that  q  (/w  /wq_/+i)  is 
both  the  swap  time  and  the  execution  time  for  each  program  and  that  qn 
is  the  overhead  time  for  each  program.    With  N  programs  at  the  particular 
level,  N  times  the  sum  of  the  swap  time,  execution  time  and  the  overhead 
time  will  determine  the  length  of  time  any  user  has  to  wait  between  execu- 
tions of  his  program. 

Equation  7  shows  that  response  time  is  directly  proportional  to  both 
the  number  of  users  and  quantum  time.    Quantum  time  has  already  been 
shown  to  have  a  lower  bound  fixed  by  efficiency  considerations  and  now 
response  time  (or  Nmax)  considerations  create  an  upper  bound.    It  is  very 
apparent  that  a  compromise  must  be  reached  between  the  number  of  users 
allowed  at  each  level  (Nmax)  and  the  required  response  time  (tr) ,  just  as 
in  the  case  of  Algorithm  A. 

A  graphical  picture  of  the  various  relationships  expressed  in  Equation 
7  is  shown  in  Figure  2 . 
3.3.2.3    Summary  of  Analysis  of  Algorithm  B 

The  following  statements  may  be  used  to  summarize  the  preliminary 
survey  of  Algorithm  B  . 

(1)  The  quantum  time  is  chosen  in  approximately  the  same  fashion  as  for 
Algorithm  A  with  the  lower  bound  set  by  efficiency  requirements  and  the 
upper  bound  set  by  maximum  allowable  user  considerations. 

(2)  Response  time  and  maximum  number  of  users  must  be  determined  by  a 
compromise  between  the  optimization  of  each. 
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(3)  All  of  the  equations  concern  one  level  only  and  consider  all  programs 
in  the  system  to  be  on  this  level  for  worst-case  analysis.    To  accomplish 
a  proper  analysis  considering  the  users  spread  about  or      ultilevels  is  a 
complex  undertaking  and  is  best  left  to  a  simulation  study  . 

(4)  It  is  possible  for  users  with  large  programs  to  get  pushed  up  to  a  high 
level  and  never  get  to  run  at  all  if  there  exist  many  small  users  with 
relatively  short  times  between  I/O  operations  .    This  could  be  corrected 
by  insuring  that  every  user  in  the  system  was  allowed  to  run  at  least  once 
during  some  cyclic  time  regardless  of  his  level  position „ 

(5)  The  large  user  program,  which  has  fast  reaction  time  (time  between 
l/0°s)  is  unduly  restricted  by  its  size  as  it  can  never  go  to  a  lower  level 
than  the  one  it  entered  upon  and  may  use  only  a  small  portion  of  its 
alloted  execution  time ,  while  still  having  to  wait  the  full  tj.  specified 

by  its  high  level  before  its  next  execution  turn*    There  should  exist  some 
compromise  between  raising  the  level  for  larger  and  longer  running  programs 
and  lowering  it  for  programs  that  consistantly  terminate  their  execution 
time  early. 

(6)  The  background  user  approach  represents  a  goodly  increase  in  efficiency 
particularly  if  the  type  of  foreground  users  are  those  who  require  periods 

of  thought  process  between  I/O  operations  and  thereby  leave  a  fair  amount 
of  idle  time  for  use  by  the  background  user. 
3.3.3    Algorithm  C 

There  exists  a  need  for  a  scheduling  algorithm  that  can  handle 
efficiently  a  variety  of  user  types  such  as:  (I)  production  job  user, 

32 


(2)  man-machine  communications  user  and  (3)  sample-data  control 
user. 

A  production  job  type  user  is  typified  by  his  ability  to  encounter 
fairly  long  delays  in  obtaining  service  without  harmful  effects  . 

The  man-machine  communications  user  needs  to  have  a  response 
time  somewhere  close  to  the  human  reaction  time ,  but  is  not  hurt  badly 
if  he  misses  a  turn  or  two  occasionally. 

Sample  data  control  users  are  very  critical  concerning  response 
time.    These  users  must  be  assured  of  a  certain  fixed  (for  each  different 
user)  interval  between  its  samples  or  execution  turns.    This  puts  an 
extreme  strain  on  the  scheduler  and  requires  a  very  stringent  priority 
queuing  scheme . 

The  task  of  trying  to  serve  all  three  of  these  types  of  users 
necessitates  the  design  of  a  scheduler  that  tries  to  meet  all  three  of 
the  goals  outlined  in  Section  2.2.    Of  course,  any  attempt  to  maximize 
three  different  forms  of  service  as  one  is  bound  to  result  in  a  system  that 
compromises  and  does  not  fully  maximize  each  individual  type  of  service. 
This  is  to  be  expected  and  resultant  service  degredation  to  any  one  type 
of  user  must  be  accepted  as  the  price  required  for  a  generalized  rather  than 
specialized  system. 

The  algorithm  proposed  for  this  generalized  type  of  system, 
Algorithm  C4  is  a  three-level  scheduling  algorithm,  where  the  various 
levels  are  defined  by  the  type  of  users  assigned  to  each  one.    The  levels 
and  their  associated  programs  have  inherent  priorities  and  reside  in 
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FIGURE  3 
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"For  Algorithm  C 


level 
and 
■priority 

user  type 

waiting 
storage 

I 

sample-data  control    ■ 

core 

II 

man-machine  communications 
(I/O  stations) 

drum 

III 

production  .jobs 
(background) 

disc 

35 


different  types  of  storage  when  awaiting  service.     Figure  3  illustrates 

the  level  organization. 

3.3.3.1    Operation  of  Algorithm  C 

The  operation  of  the  scheduling  algorithm  is  as  follows; 

(1)  At  the  beginning  of  a  response  cycle,  the  number  of  user  requests 
for  each  level  is  determined  (only  one  user  each  from  levels  I  and  III  are 
allowed  per  cycle)  . 

(2)  The  sample  rate  and  execution  time  of  the  level  I  user  will  have  to  be 
known  and  the  quantum  time  (q^)  will  be  equal  to  or  slightly  greater  than 
his  execution  time,  just  as  long  as  the  reciprocal  of  the  sample  rate  (ts) 
is  an  integral  multiple  of  the  quantum  time.,    This  is  in  order  to  be  a!  le 
to  insure  the  level  I  user  of  service  at  exactly  his  specified  sample  rate  „ 

(3)  The  total  execution  time  of  the  level  I  user  is  subtracted  from  the 
cycle  time  (which  is  also  an  integral  multiple  of  calculated  q,)  and  the 
remaining  time  is  allocated  to  level  II  and  III  users  on  the  basis  of  a  q2 
(integral  multiple  of  q-^)  basis.    The  quantity,  qn ,  is  not  allowed  to 
exceed  ts . 

(4)  Only  one  q?  is  given  to  the  level  III  user  unless  there  are  not  enough 
level  II  users  to  fully  utilize  the  allotted  time,  in  which  case,  the  level 
III  user  is  given  the  remaining  time . 

(5)  Early  terminations  of  q2  by  level  II  users  I/O  operations  are  not. 
allowed  to  change  the  structure  of  the  queue  nor  cause  recalculation  of 
qj  as  this  would  result  in  the  inability  to  insure  the  level  I  user  of  a 
specified  sampling  rate. 
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(6)  If  a  level  I  user  is  requesting  service,  a  new  cycle  execution  may 
not  begin  except  at  the  time  specified,  by  the  level  I  users  sample  rate, 
for  the  next  level  I  user  sample. 

Figure  4  shows  the  manner  in  which  the  queue  is  formed  for 
Algorithm  C  <, 

(7)  The  queue  should  both  begin  and  end  with  a  level  I  user  in  order  that 
overhead  time  involved  in  setting  up  the  queue,  etc    ,  will  occur  between 
level  I  samples  and  therefore  not  unduly  degredate  sampling  rates . 

As  the  number  of  users  from  levels  I  and  III  are  restricted  to  one 
each  per  response  cycle,  it  is  fairly  easy  to  express,  in  equation  form, 
a  relationship  concerning  the  number  of  level  II  users.    This  equation 
is  specified  by  the  following  parameters: 

(1)  tc  -       time  of  one  response  cycle. 

(2)  n  -         percentage  of  overhead  per  cycle. 

(3)  N£  -      number  of  level  II  users  . 

(4)  N^  -      pseudo  number  of  level  I  users  or  sample  rate  divided 

by  cycle  time  (l/tstc) . 

(5)  t2  -       time  to  transfer  an  average  level  II  user  to  or  from  core 

memory . 

(6)  tg  -       time  to  transfer  an  average  level  III  user  to  or  from  core 

memory . 

(7)  q«  -       quantum  time  for  level  II  users  (Mq,  ,  where  M  is  an 

integer) . 
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The  number  of  level  II  users  allowed  (N«)  is  equal  to  the  amount 
of  effective  cycle  time  (tc(l-n)) ,  minus  the  time  to  transfer  user  programs 
during  the  cycle  {N22t2=2t3,  N3=l),  divided  by  the  level  II  quantum  time 
(Mq-jih  minus  the  number  of  level  III  users  (1)  and  the  pseudo  number  of 
level  I  users  prorated  to  a  number  utilizing  a  level  II  quantum  time  (Nj/M) 
From  this  description  of  the  number  of  level  II  users  comes: 

Equation  8 

N2  =  tc(l-n)-N22t-2-2t3         _(i+Nl/M) 
Mqj, 

or 

Equation  9 

N2  =  t     (l-n)  =  2t3-q1(M+N1) 
Mq!+2t2 

3.3.3.2    Effects  of  Varying  Basic  Parameters  of  Algorithm  C 

As  might  be  expected  from  the  description  of  the  operation  of  the 
three  level  scheduling  algorithm,  the  specifications  concerning  the  level  I 
user's  needs  dominate  the  algorithm.    This  is  reflected  in  Equation  9  also. 
Note  that  the  factor  q^M  is  equal  to  the  time  between  level  I  user  samples 
(t.) ,  the  term  Nj  is  equal  to  l/tstc  and  q^  is  in  reality  the  execution  time 
of  the  level  I  user.    This  leaves  only  the  transfer  times  (t2  and  tJ  and 
the  cycle  time  that  may  be  varied  to  maximize  N2-    The  transfer  times 
are  essentially  hardware  limited  variables  and  therefore  are  costly  to 
change.    Cycle  time  may  be  increased  to  allow  more  level  II  users,  but 
will  cause  a  corresponding  increase  in  level  II  users  response  time  as  t 
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is  the  response  time  for  all  but  level  I  users .    In  short  the  critical 
factors  in  this  algorithm  are  controlled  by  the  level  I  user  and  variations 
allowed  are  few.    This  will,  in  general,  be  the  case  for  any  algorithm  that 
concerns  itself  with  control  type  users,  as  there  is  very  little  flexibility 
in  their  service  needs  . 

There  are  several  limiting  factors  inherent  in  this  algorithm  and 
the  services  that  it  can  provide.    First  of  all  consider  the  limitation  on 
the  maximum  sample  rate  allowed  the  level  I  user„    One  of  the  factors 
connected  with  the  sample  rate  limitation  is  the  average  time  necessary 
to  transfer  a  level  II  user  to  core  and  let  him  accomplish  some  worth 
while  computation  „ 

With  a  fairly  fast  computer  having  two  high  speed  I/O  channels 
and  capable  of  having  several  programs  in  memory  at  one  time,  this 
time  may  be  controlled  to  where  it  is  not  the  limiting  factor .    With  the 
above  mentioned  capabilities  the  transfer  times  are,  on  the  average, 
effectively  "swallowed"  by  the  quantum  of  computational  time  due  to  the 
allowance  of  concurrency  of  compute ,  input  and  output  operations  „ 

The  main  limiting  factor  then  comes  to  be  the  overhead  time  between 
response  cycles  as,  with  the  level  I  user  having  a  quantum  of  execution 
time  at  the  beginning  and  the  end  of  the  cycle,  the  overhead  time  effec- 
tively represents  the  sample  time  .    It  is  estimated  that  this  value  could 
be  kept  in  the  neighborhood  of  five  milliseconds  for  computers  with 
average  instruction  times  of  around  three  microseconds  ■    This  would  allow 
200  samples  per  second  or  effective  sampling  of  100  cycle  per  second 
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functions .    A  goodly  number  of  control  problems  may  be  specified  within 
this  frequency  range  and  investigated  under  multiprogramming. 
3  o  3 .  3 .  3    Limitations  of  Algorithm  C 

The  major  drawback  of  the  algorithm  is  that,  as  shown  above, 
two  high  speed  I/O  channels  ,  extensive  and  flexible  memory  protection 
and  large  core  memory  is  required  in  order  to  achieve  decent  service 
capabilities  for  the  level  I  users.    These  requirements  are  extremely 
expensive  and  are  beyond  the  reach  of  many  installations.    One  further 
drawback  is  the  reduction  of  the  level  II  and  III  users  quantum  time  in 
order  to  gain  better  sample  rates  for  the  level  I  user. 
3.3.3.4    Proposed  Changes  to  Algorithm  C 

A  possible  solution  to  the  problem  would  involve  forming  the  cycle 
for  the  level  II  and  level  III  users  alone  and  then  allowing  the  level  I  user 
to  interrupt,  when  ever  and  for  as  long  as  he  needs,  via  a  separate  quantum 
clock.    As  the  level  I  user  would  normally  not  need  a  very  long  quantum, 
this  would  not  increase  the  cycle  time  beyond  reason  and  would  allow  the 
level  II  and  III  users  to  stay  in  core  long  enough  to  achieve  meaningful 
computation  „    In  addition  the  extra  high  speed  I/O  channel  would  not  be 
needed,  as  with  a  larger  q2  the  transfer  times  could  be  accomplished, 
for  the  most  part,  concurrently  with  computation  and  would  not  degredate 
level  I  user  service  in  any  case.    A  further  advantage  is  that  computa- 
tional efficiency  would  go  up  due  to  the  overhead  involved  in  transfers  „ 
etc.-  becoming  a  smaller  percentage  of  q_ . 
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3  o  3  o  3 « 5    Summary  of  Analysis  of  Algorithm  C 

It  should  be  apparent  now  that  the  problems  involved  in  including 
capability  to  service  one-line  control  type  users  in  a  multiprogramming 
system  are  many  and  no  easy  solution  exists  that  will  provide  good 
service  for  all  types  of  users.    Probably  the  easiest  solution  to  implement 
is  to  provide  two  separate  algorithms  (one  for  all  three  levels  of  users 
and  one  for  level  II  and  III  users  only)  for  the  system  that  may  be  switched 
in  or  out  depending  whether  or  not  control  type  users  are  scheduled  for  the 
particular  time  concerned  or  not.    This  though  requires  strict  scheduling 
of  blocks  of  time  for  different  types  of  users  and  should  be  considered 
an  interim  measure  only. 

The  algorithms  involved  in  the  on-line  control  class  are  sufficiently 
complex  to  be  very  hard  to  analyze  by  simple  methods  and  therefore  lend 
themselves  readily  to  analysis  by  simulation  methods.    The  main  concern 
in  the  simulation  analysis  should  be  determining  what,  values  for  the  para- 
meters of  the  algorithm  in  question  give  the  best  level  I  user  service 
(with  respect  to  sample  rate)  and,  at  the  same  time,  the  least  degredation 
to  level  II  and  III  user  service.    The  investigation  should  be  concerned 
with  effects  on  computational  efficiency  also. 

There  exists  a  great  need  for  work  in  this  area  of  multiprogramming 
as  universities,  research  laboratories  and  even  industry  would  benefit 
greatly  by  being  able  to  solve  all  of  their  computing  needs  (control  needs 
included)  with  one  computer  system  and  yet  allow  each  user  to  feel  that 
he  has  the  system  at  his  disposal  at  any  time. 
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3 . 4      Detailed  System  Study  by  Simulation 

As  stated  earlier,  the  simulation  approach  to  the  analysis  of  a 
particular  scheduling  algorithm  is  characterized  by  its  detailed  picture 
of  the  operations  of  the  scheduler  and  its  ability  to  provide  a  method 
that  allows  the  analysis  of  an  algorithm  with  a  high  degree  interdependence 
between  its  parameters  . 

The  algorithms  mentioned  in  section  3.3  all  contain  a  certain  degree 
of  interdependence  among  their  parameters  and,  while  a  heuristic  approach 
may  bring  forth  much  valuable  information,  a  detailed  study  of  the  opera- 
tions of  the  algorithms  should  be  conducted  by  simulation  techniques  before 
decisions  concerning  changes  or  acceptances  are  made  . 
3.4.1     Features  of  The  Simulator 

The  simulator  used  for  the  purpose  of  this  paper  is  very  flexible  and 
can  be  used  to  examine  any  part  of  the  system's  operation  that  may  be 
desired. 
3,4.1.1    Input  Data  Features 

The  input  data  consists  of  such  things  as: 

(1)  User  distributions  as  pertain  to  job  arrival  times  and  various  service 
rates  . 

(2)  Job  types . 

i)  Number  of  user  channels. 

(4)  Type  of  scheduler  to  be  used. 

(5)  Parameter  of  algorithm  to  be  varied . 

(6)  Type  of  output  desired. 
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(7)  Length  of  run  (time) . 

An  example  of  the  input  data  to  the  simulator  may  be  seen  at  the  end  of 

the  program  contained  in  Appendix  II . 

The  part  of  the  input  data  concerning  the  user  distributions  and  job 
types  are  converted  by  the  simulator  to  job  specifications  for  each  user 
station  „    The  specifications  contain  the  following  information: 
lob  numper-chrono logical  number  of  the  job  generated. 

(2)  Clock  time -the  time  of  generation  of  the  job, 

(3)  Station  number- the  number  of  the  user  station  for  which  the  job  was 
generated . 

(4)  lob  type -the  type  of  job  generated  (specifies  the  particular  set  of 
r  eans  used  to  generate  the  job) . 

(5)  Arrival  time -the  time  that  the  job  will  request  service. 

(6)  Load  time -the  amount  of  time  needed  to  load  the  job. 

(7)  Active  time-the  amount  of  time  the  job  should  spend  actually  computing 

(8)  I/O  time-the  amount  of  time  the  job  will  spend  in  an  I/O  phase, 

(9)  Repeats -the  number  of  times  the  active  and  I/O  phases  will  be 
repeated „ 

(10)  Size-the  number  of  memory  cells  required  to  contain  the  jobo 
An  example  of  the  generated  job  data  is  contained  in  Figure  6. 

3 . 4 . 1  „  2    Output  Data  Features 

The  output  data  may  be  arranged  in  convenient  tabular  form  or  in 
graphical  form  depending  on  which  is  desired.    This  capability  allows  the 
investigator  wide  latitude  in  choosing  the  data  format  that  will  be  easier 
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to  analyze  for  each  output  parameter . 

It  will  be  possible  to  form  comparisons  between  various  scheduler  5 
using  the  same  user  distributions  and  determine  which  one  best  suits  the 
needs  of  the  system  and  its  expected  users  by  use  of  the  simulator „    The 
simulator  also  allows  the  study  of  the  effect  of  varying  any  algorithm 
parameter  and  can  present  tabulated  or  graphical  output  data  for  ease  of 
comparison  between  various  values  of  the  parameter  in  question  „ 

For  analysis  of  scheduler  operations,  the  simulator  output  data 
contains. 

Vai  ;able  parameter-Values  of  the  parameter  that  is  varied  during  the 
run . 

(2)  Cycle  count -The  number  of  response  cycles  that  occurred  . 

(3)  Average  cycle  time  -The  average  amount  of  time  taken  by  a  response 
cycle . 

(4)  Average  number  in  queue-The  average  number  of  active  users  durin. 
respon.se  cycle  „ 

(5)  Average  quan  turn -The  average  amount  of  time  taken  by  one  qu  mtum, 

(6)  Average  overhead-The  average  amount  of  time  taken  up  by  scheduler 
overhead  during  one  cycle - 

(7)  Computational  efficiency -The  average  amount  of  time  spent  actually 
computing  per  response  cycle,  divided  by  the  average  cycle  time, 

(8)  Average  exchange  overhead-The  average  amount  of  exchanger  overhead 
per  response  cycle. 
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(9)  Average  exchange  time-The  average  amount  of  time  taken  up  by  loading 
and  exchanging  per  response  cycle  (does  not  include  overhead) . 

(10)  Average  overload -The  average  number  of  jobs  that  were  unable  to 
obtain  service,  due  to  the  queue  being  full,  per  cycle. 

(11)  Maximum  quantum-The  maximum  amount  of  time  taken  by  one  quantum. 

(12)  Maximum  cycle  time-The  maximum  amount  of  time  taken  by  one  rev 
ponse  cycle . 

(13)  Maximum  number  in  queue-The  maximum  number  of  active  users  durin  : 
one  response  cycle. 

(14)  Maximum  overhead-The  maximum  amount  of  scheduler  overhead  during 
one  response  cycle 

(15)  Maximum  overload-The  maximum  number  of  jobs  unable  to  obtain 
service  during  one  response  cycle  . 

Maximum  number  of  stations-The  maximum  number  of  user  stations 


allowed. 

(17)  Requests  serviced-The  number  of  user  requests  for  service  honored „ 

Figure  7  contains  an  example  of  the  output  format  of  the  simulator . 

As  may  be  seen  the  simulator  output  easily  furnishes  enough  informa- 
tion for  the  proper  analysis  of  the  operation  of  a  scheduling  algorithm  „ 

For  further  and  more  detailed  information  concerning  the  operation 
of  the  simulator  consult  Appendix  I. 
3.4.2    Example  of  Simulator  Operation 

In  order  to  demonstrate  the  capabilities  and  operation  ol  the  itor, 

an  analysis  of  the  scheduling  algorithm  (Algorithm  A)  used  by  Sy  stei 
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Development  Corporation  in  their  "Time-Sharing  System"  at  Santa  Monica, 
California  will  be  conducted  using  the  simulator.    A  flow  diagram  of  t'r  e 
scheduler  section  is  contained  in  Figure  5. 

3 . 4 . 2 . 1  Input  Data 

The  input  data  is  based  on  observations  of  the  operations  of  the 
"Time -Sharing  System"  and  while  the  data  is  not  the  result  of  a  thorough 
statistical  study  of  the  system .,  it  will  be  sufficient  for  the  purpose  of 
this  example . 

3.4.2.2  Variations  of  the  Algorithm  To  Be  Considered 

The  preliminary  analysis  of  Algorithm  A  in  section  3.3,1  outlined 
several  variations  of  the  algorithm  that  would  be  worth  while  investigating 
by  simulation  techniques  „    These  and  others  will  be  analyzed  by  the  use 
of  the  simulator  and  it  will  be  determined  if  the  variations  would  further 
the  achievement  of  the  system's  goals  or  not. 

The  variations  to  be  investigated  are: 

(1)  Not  allowing  an  early  termination  of  a  program's  allotted  quantum  to 
cause  a  recalculation  and  redistribution  of  the  remaining  response  cycle 
time.    Early  termination  means  the  termination  of  a  job's  active  time  by 
I/O  operations  or  quitting  before  the  normal  quantum  is  complete 

(2)  Permitting  background  user  type  jobs  to  be  run  when  nothing  else  is 
active . 

(3)  Varying  the  maximum  number  of  user  stations  allowed  in  order  to 
determine  the  best  level  of  operation 
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FIGURE  5 
Flow  diagram  of  the  scheduler  section 
of  the  simulator 
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(4)  Varying  the  rate  of  arrival  of  job  requests  by  varying  the  arrival  time 

average  in  order  to  check  on  the  overload  conditions 

The  'variations  will  be  brought  about  by  both  changes  to  the  algorit 
and  changes  to  the  input  data  „ 
3.4.2    3    Effects  of  Early  Quantum  Termination 

In  order  to  analyze  this  problem,  the  simulator  was  used  to  simulate 
a  one  hour  normal  run  of  the  "Time-Sharing  System"  with  its  normal  type 
of  user  activity  .    A  one  hour  run  each  was  made  both  with  early  termina- 
tions  allowed  to  cause  redistribution  of  remaining  cycle  time  and  without 
this  capability.    The  input  data  was  the  same  for  each  run  and  therefore 
each  run  was  made  within  the  exact  same  user  request  environment,,    An 
example  of  the  job  data  is  contained  in  Figure  6. 

As  the  goal  for  Algorithm  A  is  to  give  the  best  possible  service  to 
as  many  users  as  possible,  the  output  data  must  be  analyzed  with  this 
in  mindo    The  run  that  accomplishes  this  goal  best  will  determine  the 
variation  to  be  recommended  for  acceptance . 

The  first  line  of  each  section  of  Figure  7  is  the  output  data  froi 
simulator  run  one  where  early  termination  has  control  over  redistributing 
cycle  time ,  while  the  second  line  of  each  section  is  from  run  two ,  where 
early  termination  does  not  have  control  „ 

An  examination  of  the  data  contained  in  Figure  7  shows  that  run  one 
honored  more  service  requests ,  gave  a  larger  average  quantum  and  had  a 
better  computational  efficiency  than  run  two  =    The  only  advantage  run  two 
had  over  run  one  was  the  decrease  in  average  scheduler  overhead ,  which 
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was  offset  by  the  fact  that  the  scheduler  overhead  represents  but  a  small 

part  of  the  total  overhead  (scheduler  overhead,  exchanger  overhead  and 
exchange  time)  and  therefore  the  total  decrease  was  ne  jligil  le 

The  outcome  of  the  simulator  analysis  clearly  shows  that  the  va:     . 
tion  of  the  algorithm  that  allows  early  termination  to  cause  a  redis*.    !  i 
of  remaining  response  cycle  time  appears  to  give  tetter  service  to 
*ve  users  and  therefore  should  be  implemented  within  Algorithm  A. 
3 o 4 o 2 „ 4    Effects  of  Allowing  Background  Users 

A  simulated  one  hour  run  (run  one)  was  made  by  the  simulator 
without  background  users  and  another  (run  three)  was  made  v»,  I  [       ick- 
ground  users  in  order  to  determine  if  background  users  should  Le  allowed 
or  not.    Lines  one  and  three  of  each  section  of  Figure  7  represent  the 
simulator  output  for  runs  one  and  three  respectively  „ 

The  data  shows  that,  while  the  rvurber  of  service  requests  re  no  red 
while  allowing  background  users  was  not  as  high  as  while  not  allowing 
them,  the  reduction  was  small  and  the  computational  efficiency  went  up„ 
That  the  computational  efficiency  went  up  only  slightly,  is  in  part  due 
to  the  system  already  being  used  very  efficiently  (for  a  multiprogramming 
system)  and  therefore  not  leaving  much  inactive  time  for  the  bac  1  ground 
user  to  take  advantage  of  „ 

It  is  recommended  that  background  users  be  allowed ,  even  though 

the  efficiency  gain  is  slight,  in  order  that  large,  long  computing  jobs 

(such  as  compilations)  may  be  run  as  background  and  not  degredate  the 

other  users'  service  as  much  as  they  would  if  they  were  run  as  regular 

time-sharing  jobs0 
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3.4.2.5    Effects  of  Varying  Maximum  Number  of  Users 

To  study  the  effect  of  variations  in  the  number  of  users  allowed, 
ten  one  hour  runs  were  simulated.    The  number  of  users  for  each  run  wa^ 
incremented  by  five  at  the  completion  of  each  run.    The  range  of  users 
allowed  was  from  5  to  SO. 

As  the  variable  parameter  was  the  number  of  users,  the  number  for 
each  run  may  be  found  in  the  variable  parameter  column  of  the  first  section 
of  Figure  8  as  well  as  in  the  maximum  number  of  stations  column  of  the 
second  section  of  Figure  8  . 

As  was  expected  s  the  results  of  the  simulation  study  showed  that, 
as  the  number  of  user  stations  allowed  went  up,  the  total  overhead  went 
up,  the  computational  efficiency  went  down  and  the  response  time 
(average  cycle  time)  increased,, 

Due  to  the  advisability  of  maintaining  a  response  time  close  to 
the  human  reaction  time  (2  seconds) ,  the  simulator  results  show  that  a 
limit  of  20  should  be  set  on  the  number  of  user  stations  allowed. 
3  a  4  .  2  ,  b    Effects  of  Varying  the  Rate  of  Arrival  of  Service  Requests 

The  varying  of  the  rate  of  arrival  of  service  requests  was  accomplished 
by  varying  the  average  or  mean  used  to  generate  the  times  of  arrival  of 
requests,  due  to  the  fact  that  if  the  mean  is  lowered,  while  the  numbei 
of  jobs  generated  remain  the  same,  the  rate  of  arrival  of  jobs  will  increase  . 
The  mean  was  varied  from  300  to  50  seconds.    An  hour  was  simulated  for 
each  new  mean,    The  first  run  was  made  with  a  mean  arrival  time  of  300 
seconds  and  each  succeeding  run  with  a  mean  of  50  seconds  less  than  the 

preceding  one . 
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The  output  of  the  simulator  analysis,  as  shown  in  Figure  a,  indicates 
that  the  overload  increases  as  the  rate  of  arrival  increases  with  a  cone 
sponding  decrease  in  cor  putdtional  efficiency. 

It  is  very  apparent  that  the  rate  of  arrival  of  service  requests 
greatly  influences  the  loading  of  the  system.    Fortunately,  except  for  an 
initial  surge  when  the  system  begins  operation  for  the  day,  a  multi- 
programming system  has  a  fairly  low  request  arrival  rate  due  to  the  slow- 
ness of  the  human  thought  and  action  process. 
3.4.3    General  Summary  of  Simulation  Analysis 

The  example  of  the  simulator  analysis  of  Algorithm  A  illustrated 
the  flexibility  and  effectiveness  of  simulation  analysis  and  outlined 
the  method  of  usin  j  the  simulator  „ 

The  simulator  may  be  used  to  analyze  and  compare  several  different 
scheduling  algorithms  in  exactly  the  same  manner  it  was  used  to  analyze 
and  compare  several  variations  of  Algorithm  A. 
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4  . 0      Conclusions  and  Recommendations 

An  investigation  of  several,  different  methods  of  analyzing  the 
operation  of  a  multiprogramming  scheduling  algorithm  was  conducted  in 
section  3.1c    The  advantages  and  disadvantages  of  each  rethod  were 
determined  and  a  composite  investigation  procedure  was  proposed  that 
would  fully  utilize  the  various  favorable  characteristics  of  eac  I      ethod 
and  negate  the  unfavorable  ones.    This  proposed  investigation  procedure 
as  outlined  in  section  3.2  f  was  demonstrated  by  the  example  of  the 
investigation  of  Algorithm  A  in  sections  3.3  and  3.4, 

As  shown  by  the  example,  the  use  of  the  composite  investigation 
procedure  will  bring  to  light  many  important  facts  concerning  the  open 
tion  of  a  scheduling  algorithm  and  allow  changes  to  be  made  to  an 
algorithm  or  a  new  one  adopted  without  disrupting  the  system  operation . 
The  use  of  the  investigation  procedure  insures  that  the  scheduler  investi- 
gated will  bring  about  a  close  realization  of  the  system's  goals  . 

The  technique  of  simulation  was  shown  to  be  a  very  powerful  and 
versatile  tool  for  use  in  the  analysis  of  an  algorithm's  operation.    The 
simulator  used  for  the  investigation  of  the  operation  of  Algorithm  A 
demonstrated  these  qualities  by  its  detailed  portrayal  of  a  wide  variety 
of  system  parameters  of  interest  during  the  simulated  operation  of  the 
5/srer:,     One  additional  feature  of  the  simulator,  that  was  noticed 
during  the  course  of  the  investigation,  is  the  detailed  insight  obtained 
of  not  only  the  operation  of  the  scheduler,  but  also  the  complete 
multiprogramming  system ,    This  insight  was  gained  by  the  process  of 
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formulating  and  coding  the  simulator .    This  experience  and  knowledge 
of  the  system  would  be  invaluable  to  the  system's  programmer  an  :   i 
simulation  study  of  a  system  will  pay  off  in  this  respect,  alone 

Some  of  the  major  problems  involved  in  combining  the  on-line 
control  type  user  with  the  normal  users  of  a  multiprogramming  systei 
were  outlined  during  the  heuristic  analysis  of  the  algorithm  proposed  I 
the  author  (Algorithm  C)  in  section  3. 3 .3.    The  foremost  obstacle  beir  ] 
the  inability  of  the  control  type  user  to  allow  his  service  requirements 
to  be  varied,  in  order  to  be  compatible  with  the  rest  of  the  users,,    It 
must  be  accepted  that  all  other  users  bow  to  the  requirements  of  the 
control  user  i.e.,  the  control  user's  requirements  in  effect  determine 
the  scheduler  design .    With  this  in  mind,  Algorithm  C  and  the  modifica- 
tions to  it,  as  proposed  in  section  3.,  3.3.4,  offer  reasonable  solutions 
to  the  problem  of  integrating  the  control  type  user  into  the  multiprogram- 
ming system. 

The  state  of  the  art  in  the  multiprogramming  area  may  be  briefly 
described  as  just  coming  out  of  the  hardware  limited  phase,     Hereto  "ore 
it  has  been  extremely  difficult  to  obtain  the  necessary  capabilities  in  a 
computer  system  to  allow  effective  multiprogramming.    As  a  result,  many 
features  that  should  have  been  of  a  hardware  nature  have,  Ln  some  cases 
been  implemented  by  pro  n  an  ming  (with  a  resulting  increase  in  overhe 
or  have  been  omitted,    A  few  of  such  features  are: 

(1)  Priority  assignment  of  interrupts  and  users. 

(2)  Quantum  determination  and  interruption  „ 


(3)  Memory  protection  0 

With  the  new  large  computers  being  designed  for  multiprogramming 
applications,  this  state  of  affairs  will  cease  to  exist  and  the  simple 
scheduling  algorithms  of  today  will  have  to  be  replaced  by  more  complex 
and  versatile  ones.    The  algorithms  in  present  use  are  entirely  sati  •'  t<  *ory 
for  the  present  requirements  of  20  to  50  user  stations  utilizing  Teletypes 
and/or  displays  as  1/0  devices  and  operating  in  such  a  manner  as  to  recpire 
only  a  small  portion  of  their  time  in  the  system  to  be  compute  time 

The  present  scheduling  philosophy  is  capable  of  providii    \    ilgorithms 
for  use  within  systems,  that  concern  themselves  primarily  with  satisfying 
a  fairly  large  number  of  users,  whose  needs  are  short  bursts  of  compute 
time,  followed  by  longer  intervals  of  idle  or  I/O  timeo    A  change  in 
philosophy,  however,  appears  to  be  necessary  in  order  to  provide  adequate 
schedulinq  algorithms  for  use  within  the  military  command  and  control 
type  of  multiprogramming  system,  where  the  numerous  tasks  and  service 
requirements  are  both  extremely  complex  and  exacting .    This  change  also 
appears  necessary  when  considering  the  type  of  system  that  contains 
multiple  facilities  (such  as  multiple  processing  units,  etc.)  and  requires 
'he  scheduler  to  schedule  the  use  of  each  and  every  one  of  the  multiple 
units.    These  problems  are  too  complex  for  solution  by  present  scheduler 
philosophy  and  represent  areas  of  multiprogramming  that  are  fen-!e  :«el 
for  future  research. 

Further  research  into  the  use  of  simulation  techniques  for  analysis 
of  scheduling  algorithms  is  needed  as  the  capabilities  of  simulation  are 
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enormous,  if  utilized  properly  and  integrated  with  other  techniques  in 
the  correct  manner . 

A  final  recommendation  is  that  the  applicability,,  of  dynamic 
programming  to  the  scheduler  problem,  be  investigated  thoroughly,  as 
the  analysis  of  a  scheduler  is  a  form  of  cost-effectiveness  study  and 
dynamic  programming  is  very  effective  in  this  area. 
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APPENDIX  I 
SIM-A  MULTIPROGRAMMING  SYSTEM  SIMULATOR 

Simulation  has  become  a  valuable  analytic  method  that,  can  be 
applied  in  diverse  situations.    The  type  of  simulation  of  interest,  is  the 
Monte  Carlo  Method  which  is  defined  in  the  McGraw  Hill  Encyclopedia 
of  Science  and  Technology,  as  "A  technique  for  estimating  the  solution, 
x,  of  a  numerical  mathematical  problem  by  means  of  an  artificial  sampling 
experiment  .    .    . "    The  method  aptly  fits  the  multi programming  system 
problem  and  can  produce  worthwhile  results .    The  required  probability 
distributions  associated  with  users  can  be  determined  by  general  data 
gathering  and  observation.    The  use  of  various  algorithms  in  the  Executive 
routine  and  several  hardware  configurations  in  a  simulated  system  subjected 
to  a  typical  loading  will  produce  the  data  to  obtain  a  measure  of  effective- 
ness for  the  various  hardware  and  software  configurations.    The  time  and 
expense  required  to  actually  evaluate  each  of  the  combinations  in  an 
operating  system  are  prohibitive  and  the  use  of  simulation  techniques 
provides  the  only  realistic  approach. 

Program  SIM  was  developed  as  a  general  multiprogramming  system 
simulator  with  the  emphasis  on  the  time-sharing  type  of  environment, 
Due  to  the  specific  nature  of  the  authors'  theses,  primary  attention  was 
given  to  the  Scheduler  and  Exchanger ,    The  normal  performance  of  other 
specific  areas  of  the  Executive  routine  is  assumed  and  these  portions 
treated  in  a  block  method.    A  prime  example  of  this  is  the  Dispatcher, 
While  it  is  a  critical  area  of  a  multiprogramming  system  no  specific 
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characteristics  were  assumed.    When  a  user  has  an  Input  or  Output  the 
program  assumes  a  waiting  status  until,  due  to  the  incrementing  of  the 
simulator  clock,  the  action  is  deemed  complete.    The  availability  of  the 
required  I/O  equipment  at  all  times  is  assumed.    A  time-sharing  system 
is  characterized  by  frequent  man-machine  communication  and  buffered 
I/O  is  usually  impossible  due  to  the  step  by  step  nature  of  the  system. 
However,  simultaneous  I/O  by  all  users,  at  least  to  their  reactive 
typewriters,  must  be  permitted .    The  gross  Dispatcher  treatment  provides 
all  this  and  only  avoids  the  complications  of  particular  operations  ,    If 
this  area  is  studied  in  the  future  the  Dispatcher  portion  could  easily  be 
made  more  detailed  and  added  to  the  system  simulator,, 

The  job  load  on  the  simulated  system  is  created  by  a  job  generation 
subroutine  (SET)  „    Each  job  is  characterized  by  six  variables,  which 
define  any  job  entering  the  system.    Arrival  time,  the  first  parameter, 
is  assumed  to  be  exponentially  distributed  on  the  basis  of  queuing  theory 
concepts  and  actual  observation  at  System  Development  Corporation.    A 
variable  parameter  is  the  mean  arrival  time  expressed  in  seconds  „    The 
value  of  arrival  time  was  determined  by  taking  the  natural  logarithm  of  a 
uniformly  distributed  random  number.    The  second  parameter  is  Load  time 
and  represents  the  time  required  to  transfer  the  binary  program  from  its 
permanent  storage  to  the  temporary  storage  having  access  to  the  central 
memory.    The  next,  three  parameters,  Active  time,  I/O  time  and  Repeats 
define  the  actual  program  operating  characteristics.    A  program  once 
loaded  into  the  system  is  assumed  to  have  an  active  period  followed  by 
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an  l/O  period,  during  which  no  service  is  required  from  the  central 
processor.    This  cycle  is  repeated  until  the  job  is  completed,  or  there 
are  no  further  repeats  required.    Due  to  the  nature  of  SIMf  differences  in 
I/O  such  as  tape  transfers,  searches  or  outputs  to  reactive  devices  are 
not  recognized  and  the  I/O  operations  are  grouped  together ,    The  sixth 
parameter,  Size,  completes  the  job  description .    Program  size  is  limited 
to  a  maximum  length  of  one  hundred  cells  less  than  the  full  core  available 
for  operating  programs .    The  last  five  parameters  are  deter  mined  using  a 
Gaussian  Random  number  generator  and  the  mean  values  received  as  input  . 
A  uniform  random  number  generater  is  used  to  generate  any  of  ten  possible 
job  types.    The  probability  of  each  type  job  is  also  received  as  an  input. 
As  soon  as  the  job  is  completed,  that  is  the  number  of  repeats  remaining 
is  less  than  zero,  a  new  job  is  generated  for  that  station „    An  example 
of  the  input  to  SIM  is  shown  at  the  end  of  the  program  contained  in 
Appendix  II . 

It.  is  possible  to  obtain  a  wide  variety  of  output  parameters  from 
the  simulator  as  it  has  access  to  all  of  the  internal  system  parameters 
concerning  the  operation  of  the  system  in  question „    These  parameters 
may  be  gathered  on  a  minute,  average,  total  or  maximum  basis  and 
thereby  present  a  picture  of  the  system's  operation  in  almost  any  degree 
of  detail  desired. 

For  the  purpose  of  comparison,  the  output  parameters  of  one  run, 
using  a  certain  hardware  and  software  configuration,  may  be  saved  and 
then  presented  with  the  results  of  another  run ,  with  changes  to  the 
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hardware  and/or  software  configuration.    The  output  of  the  simulator 
may  be  in  a  tabular  format  or  modified  to  a  graphical  type  format, 
which  ever  is  deemed  best  for  comparison  purposes  . 

The  program  operation  is  cyclic  in  nature .    First,  the  initial  jobs 
are  generated  and  the  program  constants  read  in  and/or  initialized  for 
the  run .    The  main  body  of  the  simulator  is  then  entered  and  the  actual 
run  commenced.    The  clock  is  checked  against  arrival  times  of  all 
allowed  users  (maximum  50}  and  an  equality  or  greater  than  condition 
sets  the  action  entries  of  the  status  table  {STAT(X,Y)h    The  Scheduler 
then  determines  which  requests  for  service  shall  be  honored  and  the 
order  in  which  they  shall  be  honored,  i.e.  ,  queue  formation.    The 
Scheduler  also  determines,  from  the  number  requesting  service,  the 
amount  of  time  each  user  is  allowed  per  cycle  „    The  cycle  begins  with 
the  formation  of  the  queue  and  ends  with  termination  of  the  last  user's 
quantum.    The  Scheduler  then  passes  control  to  the  Exchanger .    For 
further  specific  information  concerning  the  operation  of  the  Scheduler 
section  of  the  simulator  see  Lt„  W.  G.  Wilder's  thesis. 

The  Exchanger  determines  the  action  required  by  the  next  user 
in  the  queue  and  then  LOADS,  QUITS  or  TRANSFERS  the  users  program. 
The  actual  transfer  algorithm  is  variable  and  the  methods  used  are  dis- 
cussed in  detail  in  Lt,  R»  R.  Hatch's  thesis.    Regardless  of  the  exact 
method,  the  required  transfers  are  determined  and  the  effective  transfer 
times  (TELOAD  and  TEDUMP)  and  exchange  overhead  calculated  and 
added  to  the  clock.    In  the  LOAD  and  QUIT  operations  the  size  of 
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the  external  store,  such  as  a  drum,  is  considered  and  no  load  or  storage 
full  conditions  are  possible. 

At  the  completion  of  a  cycle  all  users  in  an  I/O  status  are  handled 
by  decrementing  the  remaining  time  by  the  elapsed  cycle  time*    Users 
completing  I/O  are  checked  for  repeats  0    If  repeats  are  necessary  the 
program  is  reset  to  the  active  mode  and  if  no  repeats  are  required  the 
program  is  terminated  by  a  QUIT  command  in  the  next  cycle. 

To  avoid  long  idle  periods  a  scheme  is  used  to  advance  the  clock 
to  the  next  active  clock  time  if  there  are  no  users  desiring  service  .    The 
smallest  I/O  time  remaining  (SMALLA)  and  the  nearest  arrival  time  (SMALLB) 
are  determined  and  the  smaller  of  these  two  added  to  the  clock  and  a  new 
cycle  commenced. 

A  maximum  clock  parameter  read  in  terminates  the  run  and  the 
capability  for  recycling  is  provided „    All  new  parameters  may  be  read  in 
or  the  original  parameters  may  be  modified  for  successive  runs. 

A  flow  diagram  of  SIM  is  contained  in  Figure  1-1  and  a  copy  of  the 
actual  program  is  contained  in  Appendix  II  „ 

Due  to  the  fact  that  the  thesis  topics  of  Lt„  W.  G„  Wilder  and 
Lt.  R„  R„  Hatch  are  both  in  the  multiprogramming  area  and  a  generalized 
multiprogramming  simulator  could  be  used  in  each  case,  this  simulator 
represents  the  joint  efforts  of  both  Lt  ,  Wilder  and  Lt..  Hatch. 
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